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ABSTRACT 

An experimental sociological research training 
program at Columbia University is described arid analyzed. Two broad 
objectives of the Columbia Program are defined: (1) to bring 
sociological perspectives tp bear on educational theory ^nd prpblems, 
with emphasis on discovering and codifying the methodological 
approaches most suitable for investigating educational systems from a 
sociological viewpoint; and (2) to prepare researchers ^ho are 
qualified to undertake major administrative responsibilities as 
directors of large-scale research projects, research programs and 
research institutes that focus; on education. .Focus of the paper is on 
description of the program which was designed to attain these 
objectives and on determination of the effectiveness of the program. 
The following aspects are examined: (1) recruitment and selection of 
trainees, (2) strjacture of the program, (3) content of the program, 
(4) personnel, and (5; evaluatio and implications. . (Author/SHM) 
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INTRODUCTION 

In the last decade a number of multidisciplinary programs for 
training educational researchers have been developed. In this 
paper I will describe and analyze one such experimental program, 
namely the USOE-supported research training progr^ in the Bureau 
of Applied Social Research (BASR) and the Department of Sociology 
of Columbia University. The Columbia program began in September 
1966 with nine 'trainees . Three trainees are currently receiving 
terminal year support. Sam Sieber of Columbia University and the 
BASR^ has directed the program since its inception. 

I was in the Columbia program from 1966-1971 (supported as a 
trainee for two years and independently supported for three years) . 
Hence my description and analysis draws heavily upon my own partici- 
pation, observation and notes. In addition, I have studied the 
Training Director's progress reports, the seminar notes and reports, 
and had numerous individual conversations with the Director and with 
other trainees during the past eight years. 

As stated by the Training Director (Sieber, 1971, p.l), the 

program has two broad objectives: 

(1) to bring sociological perspectives to bear on 
educational theory and problems, with emphasis 
on discovering and codifying the methodological 
• approaches most suitable for investigating edu- 
cational systems from a sociological viewpoint; 
and 
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(2) to prepare researchers who are qualified to 

undertake raajor administrative responsibilities 
as directors of large--scale research projects, 
research programs and research institutes that 
focus on education. 

What type of program was designed to attain these objectives, and 
how effective is that program? This paper will focus on these two 
questions, even through we do not have the type of comparative 
evidence that would demonstrate the relative effectiveness or ineffec- 
tiveness of this program. We will examine the following a^spects of 
the Columbia Program: (l)- recruitment and selection, (2) structure 
of the program, (3) content of the program, (4) personnel, and 
(5) evaluation and implications. 

RECRUITMENT AND SELECTION 

Trainees were recruited from the pool of entering and already 
enrolled graduate students in the Department of Sociology at Colvunbia. 
The Training Director participated in the selection of entering 
students for financial aid, and thereby identified potential candi- 
dates for the Training Program. The criteria for selection of 
trainees were the following: Graduate Record Exam scores; academic 
grades; letters of reference; course papers; an interest in education 
as a sociological specialty; a demonstrated or strongly expressed 
interest in empirical research; and an intention to fulfill require- 
ments for the Ph.D. degree. Several of these criteria were determined 
from application materials, while interests and intentions were 
ascertained by the Training Director's personal interview with candi- 
dates. Through discussion with other members of the faculty who.- 
participate in the allocation of financial aid, a list of candidates 
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for the program was drawn up. These students were then contacted 
^ by the Training Director for a screening interview. The Training 

I' Director made his decision on the basis of interviews and written 

I application materials. (Sieber, 1971, p. 2). 

I In addition to formal criteria, two other factors affected the 

i 

recruitment of trainees. One factor was the prestige of the Program 
within the Department and the other was the scarcity of fellowships 
. for graduate students. 
! Since there were more graduate students than available fellow- 

ships or assistantships in the sociology department,, the Training 
! Program was able to be selective in its appointments. Thus the 

1 students in the program were above average academically. The size 

\ of the stipends and the high caliber of the students in the Program 

i 

j helped to offset the generally low prestige of sociology of educa- 

I tion in the Columbia Department of Sociology. (Prior to the Train- 

1 Ing Program, sociology of education was not offered in the Depart- 

I ment) . 

I Just as the relative prestige of a program in a departn^ent 

I affects recruitment, the scarcity of fellowship funds impinges on 

selecting trainees for a program. As other fellowships for Columbia 
graduate students began drying up, the department would sometimes 
urge the Training Director to support a particularly promising 
student, regardless of the student's interest in doing research on 
education. To the extent that this influence brought some of the 
I best students into the Program, it benefited the other trainees and 

I further enhanced the Program's prestige. It also created an oppor- 
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tunity for converting uninterested students into sociologists of 
education. Where these two positive outcomes did not occur/ these 
contextual pressures served to push the Program away from its objec- 
tives. At Columbia there were so few cases and so little data that 
it is impossible to draw conclusions about this particular problem. 
I suspect that this issue would arise in any situation where fellow- 
ship resources were scarce , however. 

In short, the case of Columbia shows the importance of stated 
formal criteria in selecting trainees, but also suggests that cer- 
tain contextual characteristics such as the relative prestige of 
a program at a particular school and the scarcity of other fellow- 
ships may affect the recruitment process. 

STRUCTURE OF THE PROGRAM- 

The Columbia Program is characterized by integration and coopera- 
tion, both within the seminar and in relation to its context. One 
of the integrative aspects of the Program is a weekly training seminar, 
where beginning and advanced students regularly meet together. 

The value of ha\dng a structure that is not age**graded is illus- 
trated in the way the seminar members help prepare, each other for 
comprehensive oral exams. These orals have traditionally been a 
major hurdle in the path to a Columbia Ph.D. in sociology. Many 
students not integrated into a training program postpone taking orals 
for two, three or even four years after they have completed the first 
two years of course work. In the Training Program seminar one or 
two sessions a year are devoted to giving practice orals to students 
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about to face their examiners and to haydng successful candidates 
report back after the experience. They describe what it was like, 
what type of studying was most helpful, what types of questions were 
asked. This opportunity provides institutionalized rather than hap- 
hazard anticipatory socialization. It also serves to make successes 
juore visible than failures. At Columbia generally the horror stories 
were more evident than the success stories. By having a seminar of 
beginning and advanced students together, younger students see models 
close to themselves who are passing orals, doing dissertation re- 
search, successfully defending their dissertations, thus giving them 
a much clearer and more positive sense of what the career of a 
graduate students is than they would otherwise have. 

A necessary condition for the success of such a seminar seems to 
be having a cooperative rather than a competitive ambiance in the 
seminar. Two features of the Columbia Program served to minimize 
^competition and a third enhanced cooperation. First, unlike other 
courses at Columbia, students received a pass/fail grad.^ rather than 
a letter grade in the training seminar. Thus no one gained by having 
someone else do poorly. Second, everyone in the program who v;orked 
conscientiously was assured of continued fellowship support through 
thoir fourth year. Because the rewards of grades and money were 
equal for all and the pie was large enough for everyone to get their 
piece, trainees could afford to cooperate rather than compete with 
each other. The cohesiveness and cooperation of the seminar group 
was enhanced by the common tasks that they had to overcome, particu- 
larly since those obstacles were imposed by a structure outside the 
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seminar. The tasks included the department's language, mathematics, 
comprehensive examination and dissertation requirements. The excel- 
lence of any particular student benefited the group, both through ■ 
the expertise that students shared with the group, as well as 
through the enhanced prestige of the group by virtue of a member's 
performance (since most faculty members and at least some other 
students knew who was in the Program) . 

Another important aspect of the i?rogram is the informality and 
accessibility of the Training Director. Unlike other Columbia faculty 
members who are in their offices two hours a week, with their doors 
closed, the Training Director is in his office with an open door 
eight <r more hours a day. usually seven days a week. He is willing 
V to drop vrfiat he is doing and talk at length with student*? about their 
research or their careers. After the first weeks of seminar even 
beginning students call the Director by his first name. Discussions 
begun in seminar or in the office often continue later in the local 
West-End bar, where other faculty or BASR researchers join the Direc- 
tor and students in their lively discussions. This type cf informal 
socialization. contributes a great deal to the development of socio- 
logical sensibilities in budding researchers. (For a discussion of 
some of the proceoses of this informal socialization in the develop- 
ment of natural scientists see H. Zuckerman, 1965). 

In addition to its favorable interpersonal features, the Train- 
ing Program is positively affected by its location in the Bureau of 
Applied Social Research (BASR) at Columbia University, where the 
Training Director is a staff member. The Bureau has a program in 
education research, headed by the Training Director, which includes 



a number of large-scale stuaies of education. -In addition to the 
clerical r data-processing and library facilities of the Bureau # 
therefore # the trainees have daily access to professional staff who 
are doing ref:ea"rch on education. Moreover # when openings occur 
for research assistants, the trainees are given priority and strongly 
urged to participate. A major responsibility of the -Training Direc- 
tor is either to recruit trainees on staff projects or to help them 
develop proposals for funds ,for their own research at the Bureau. 
In either case# the Training D. rector continually monitors their 
work . 

Besides benefitting from the BASR's staff and facilities, 
trainees are in contact .7ith a variety of outside educational agencies, 
as a result of their current research activities. While none are 
"located" in these agencies for an extended period of time, several 
are regularly in contact with researchers in other agencies (e.g., 
R&D Centers, State Departments of Education) and with local school 
personnel; another has extensive relationships with the USOE-Edcua- 
tional Opportunity Grant division and with colleges; and two others 
have worked extensively in Indian high schools, sought advice from 
educational researchers at Teachers College (Columbia) , University 
of Alaska, Berkeley, and elsewhere, and obtained data from the Bureau 
of Indian Affairs. Other trainees have done research in cooperation 
with the Center for Urban Education; Teachers College, Columbia; the 
Washington, D.C. school system; the American Edcuational Research 
Association; the Research Training Branch, USOE: and the Bureau of 
Social Science Researcn, Washington, D. C. 



In brief, in both its interpersonal structure and its relation- 
ship to relevant organizations, the Training Program is designed 
to njaximiae learning and to provide opportunities for trainees to 
practice the roles they are preparing to play in their future careers, 

CONTENT OF THE PROGRAM 

We can organize the substance of this Program into five major 
components. These are:. (1) Knowledge of sociological research on" 
education; (2) Study of the processes and proceSJ^ies of actual re- 
. searchers; (3) Methodological experiments designed to suggest what 
is gained or lost by various methodological approaches; (4) Develop- 
ment of important research skills; and most importantly (5) Research 
apprenticeships culminating in the student's own independent research. 
We will briefly consider each of these components.. 

When the Training Program began, the Director felt that students 
needed some perspective on research in the sociology of edcuation. 
In order to teach students to recognize areas of needed research by 
themselves, for example, the Director suggested that they investigate 
the research literature, rather than passively reading summaries of 
the state of the field. They began by ascertaining what ideas for 
future research had been noted by Brim (1958) and Gross (1959), and 
then examined the Sociological Abstracts to see if those problems • 
had be^n treated in subsequent research. The trainees doubted that 
the Abstracts would provide accurate information about the^ articles . 
This exercise furnished an opportunity to note the importance of 
taking short-cuts in research that promised to be much less expensive 
in time and effort; and the seminar compared the estimated cost of 
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going to the original articles wi.\ the estimated cost of using the 
Abstracts. It was clear that the latter procedure was vastly superior 
in terms of cos t- analysis; but the question of external validity 
still perturbed the trainees. 

An exercise was therefore arranged for a future session: half 
the trainees would read articles and half would read the correspond- 
ing abstracts; they would write .Jown their impressions for the 
research from their individual source (article vs. abstract); and 
they would compare their impressions. In the words of the Training 
Director^ "This turned out to be a useful exercise for several reasons: 
it wade us recognize the limitations of uhe Abstracts (which are 
frequently used for similar purposes of content analysis) , it led to 
a discussion of the value of comparing cheaper and costlier measures 
before deciding which to adopts it helped us to design the content 
analysis and, in the case of one of the articles, it led to a leugth^: 
discussion of an important research finddng." (Sieber, 1966a, p. 3). 

Second, the Training Program seeks to study in depth the 
processes and- procedures of actual researchers. Trainees interview 
directors of completed and continuing research projects with respect 
to decisions they made in the course of their research^ with an 
effort to determine the rules or basis they used in making their 
decisions. Thus another pedological approach in the seminar involves 
the dissection of living research studies. 

Third, the Training seminar conducts methodological experiments 
to gain new understanding of the values and limitations of various 
methods. One such experiment was an exercise in the comparison and 
integration of field work and survey research. One of the researchers 
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at the BASR was directing aii evaluation study of the Urban Corps in 
New York City. In that program college '•internes'' were placed for 
summer jobs in various city agencies. • The evaluators had two kinds 
of field notes on the various agencies and had questionnaire data 
from the student internes. The trainees were instructed to study 
thoroughly the field notes on two different city agencies, and then 
to select items from the questionnaire which they felt were sus- 
ceptible to prediction on the basis of the field notes. Specifically, 
they were to select those items th . would (1) differentiate agencies 
according to the success of their interne programs, and (2) to predict 
the modal response of the Urban Corps intern es in th e two agencies, 
on each of the selected items. . The evaluator then presented the 
actual distribution of survey responses. Comparing predicted with 
actual responses disclosed that the inability to predict was due to 
(1) lack of information in the field notes, (2) irrelevancy of cer- 
tain items to predicting the success of the agency's internship pro- 
gram, or (3) failure to draw out the implications of the field notes. 
The field notes tended to provide global statements about agencies, 
and<S.d not give the distribution of responses found in the survey 
results. By showing the areas of knowledge in which field work and 
survey work did and did not overlap , as measured by the incidence and 
the accuracy of predictions, the exercise demonstrated the distinc- 
tive contributions of each technique and suggested ways the two 
sources of information might be integrated. 

Further^ since there were two sets of field notes on each of 

* 

the two agencies, one written by a staff member of the-Urban Corps 
(the client) and the other by a graduate sociology student, the 



trainees were asked to note any differences between the client's 
self-study and the sociologist's field work, and to recall which 
effort was most helpful in their predictions. The purpose of this 
was to bring out any distinctive features of a sociological field 
study as contrasted with administrative " self -surveys which are 
quite common in educational settings. The client's self-study con- 
tained a good deal of social bookkeeping, many value judgments, and 
rather sweeping recommendations, while the sociologist's notes con- 
tained many indicators of the quality of social interaction, but 
virtually no social bookkeeping or recommendations. In trying to 
gain an overall picture of the agencies, however, it turned out that 
both sets of field notes were necessary, which led to considerations 
of combining different field approaches. (See Sieber 1966a & b) • 

In addition to surveying the field of sociology of education, 
dissecting live research studies, and doing methodological experiments, 
the Training Program seeks to develop an array of other useful research 
skills including proposal writing, research administration, question- 
naire construction, field study, coding, computer technology, 
analysis of quantitative and qualitative data, and writing and editing 
research reports* These skills are taught by doing* the actual opera- 
tions, either in seminar sessions or in the research component of 
the Program. 

A singularly important feature of the Program is the way trainees 
actually conduct research studies during three or more years. They 
do this first as an apprentice on existing research projects, then 
for their dissertation they plan and conduct their own research 
studies. To illustrate, current trainees (including the independently 
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supported ones) have the following research responsibilities; 
(1) participant observation research on a new teacher organization 
set up to promote professionalism and preparation of a proposal to 
study socialization of beginning teachers; (2) research assistant on 
BASR project for evaluation of USOE Pilot State Dissemination Program? 
(3) co-project director, national evaluation of paraprofessionals in 
Follow-Through program (USOE) ; (4) research assistant on evaluation 
of Educational Opportunity. Grant Program and Work Study Program • 
(both USOE),, (attends conferences with USOE personnel, interviews 
financial aid officers, works on sample design, data analysis, etc.); 
(5) analysing survey data on determinants of teacher satisfaction in 
sxaburban schools. Eight former trainees are now doing doctoral 
dissertations I on such topics as: high school protest, its causes and 
management; teachers* definitions of success and failure of pupils; 
militance and education among coal miners in Appalacia; and adoption 
of innovations in urban high schools. Five Master's theses have been 
written on topics such as "American Indian Leadership and Secondary 
Education", "Relationships between Society and Education in Western 
Europe: 1945-1968", and "Measuring Educational Research Quality". 
Five doctoral dissertations have been completed. They are: "The 
Effects of Federal Aid to Higher Education on Social Inequality" 
(Published by Jossey-Bass as The Illusion of Equality , by Murray 
Milner) ; "A Study of Black Teachers in a Ghetto School System", (to 
be pxablished by Praeger) ; "The Quality of Research on Education: An 
Empirical Study of Researchers and Their Work"; "Social Work: The 
Case of a Semi-Profession" (an article based on it has been published 
in The Semi-Professions and Their Organization , edited by Amitai 
Etzioni) ; and "Bureaucratization and Social Control". (Sieber, 1971). 



In short, the svibstantive aspects of the Program serve to 
acquaint students with the field of education, develop their research 
sensibilities and skills, and culminate in their completing one or 
more major research studies. 

PERSONNEL 

In any description and evaluation of a social program it is 
essential to distinguish the structure and content of the program 
from the staff that is administering it. We have already noted the 
positions held by the Training Director, as well as his infoirmality 
and accessibility. It is important to add that the Director of this 
Program has another very important characteristic. He is himself a 
working researcher. Thjis he is constantly involved in designing, 
operationalizing, conducting, analysing and writing up research 
studies. It is rare to find one person who is willing to spend so 
much time with students and who also works very hard on his own 
research. Such an example may teach far more than any didactic seminar. 

EVALUATION AND IMPLICATIONS 

Perhaps the fairest way to evaluate this or any other social 
program is to compare the stated goals of the Program with the reported 
and observed outcomes. We can evaluate the outcomes of this program 
from at least four perspectives — • that of the Director, trainees, 
sponsors, and our own, as researchers. 

1 In view of the trainees* extensive involvement in research 
throughout the Program, the Training Director is "highly pleased with 
the practicum". (Sieber, 1971, p. 12). Within the Program itself he 
feels that he has accomplished what he set out to do. 
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The trainees' reports support this positive evaluation by the 
Training Director. One writes that being part of a BASR evaluation 
of a pilot project for the dissemination of research information to 
local schools "is giving me the opportunity to be part of a large- 
scale research project from its inception, and thus gain experience 
in dealing with methodological, administrative and analytical problems 
connected with educational research". (Sieber, 1971, p.l3) . Another 
reflects that his experiences as a research assistant on two very 
different projects "have been invaluable in introducing me to social 
research techniques and ideas through actual ■ participation in major 
studies". (Sieber, 1971, p.l4) . A third trainee concurs, "During 
my second year as a trainee... i began working on a study of the 
relationship of the educational experiences of Indian students to the 
development of indigenous leadership in two American Indian tribes. 
This project has provided me with the opportunity to become familiar 
with every stage of survey research. Under the guidance of the 
Program Director I have learned how to write questionnaires, conduct 
interviews, and code and analyze data". (Sieber, 1971, p.l4) . 

A third perspective for evaluation is that of the sponsors of 
the Program. Judging .from progress reports, written work produced 
by trainees and site visitations^, USOE continued to support the Program 
from 1966 until 1972-1973. I have very little data on this perspec- 
tive , however . 

We have seen that the Training Director, trainees, and sponsor 
favorably evaluated the Program being examined here, but surely that 
is not surprising. Wnat evidence would we need to convince a hard- 
nosed skeptic that the Program has accomplished its objectives? From 



a single case study such as this we cannot get the necessary evidence 
to show that this Program is better than most in attaining its p\ir- 
pose. We can, however, use this case to formulate a research design 
that might be used to evaluate research training programs in general* 
In order to show that one type of training program is relatively 
more effective than another type, we need to have evidence on three 
classes of variables, namely, the recruits entering a program; the 
structure, substance and personnel of a training program; and outcomes 
of the program. As noted earlier, recruits differ with respect to 
their interest in educational research, their intellectual abilities, 
and their other personality traits. A given program varies according, 
to its structure, cognitive and affectual content, and its context. 
All of those variables would need to be specified and measured in an 
evaluation of training programs. Personnel can differ according to 
positions held, behavior, and dedication. Outcomes may be reflected 
in the sentiments of the Training Director and the trainees and by 
the behavior of graduates, i.e., what they end up doing. The problem 
of evaluation, of course, is showing that it was the Training Program 
rather than the interests and abilities of the recruits or something 
else that determined the outcomes. Furthermore, the success of a 
program may depend upon factors beyond its scope, such as the oppor- 
tunity structure faced by graduates. The effectiveness of one type 
of program vis a vis another one can be ascertained by the relative 
success of graduates, because we might be willing to assume the 
opportunity structure as a constant for all graduates of the same year. 

Does this case study offer any insights that might be genera 1- 
izable to other training programs? Analytically it seems probable 



that the reputation of the department running the program helps in 
providing a pool from wliich to draw trainees^ although a less pres- 
tigious department cculd recruit directly from undergradiusite schools 
for a special program. Within a department, the relative prestige of 
the program is important for attracting and keeping good graduate 
students. As already noted, the prestige of a program is probably 
a function of the relative scarcity of fellowship support and of the 
reputation of the Program's director, students, and substance. The 
scarcity of stipends may raise the academic level of recruits but 
bring in some who are only marginally interested in education, depend- 
ing on the vulnerability of the program to departmental pressure. 

Given a pool of potential trainees, we need to know more about 
the predictive validity of various criteria for selecting candidates. 
It may be that such intangibles as energy, commitment, originality, 
or an integrated personality are more important for latex success as 
an educational researcher than 50 or 100 additional points on the 
Graduate Record Exam. We have no: systematic studies of what personal 
characteristics are the best indicators of future contribution to 
behavioral science research. 

Besides stressing recruitment, we have noted the importance of 
the structure of this Training Program. That observation is consistent 
with other research on the training of researchers. Studies of the 
socialization of eminent scientists, for example, have shown that 
most had a close personal apprenticeship with a master researcher, 
and in that way they developed their research sensibilities. Structur- 
ing a training experience so this could occur is possible in many 
programs. Finding tlie people who are able and willing to serve as 



exemplary models is more pr-^^blematic. * 

The substance of a training program may be its most easily 
measured aspect. VJe can determine how much beginning trainees know 
about the field of education and about actually doing research studies, 
and compare that with what they know upon completion of the program. 
But knowledge alone cannot be considered a sufficient indicator of 
the success of such a training program, because without the desire 
and the opportunity to do research, the most knowledgable graduate 
adds nothing to our understanding of the process of education. 

In a nutshell, the evaluation of any research training program 
hinges on the answers to two questions: What, and how well, do trained 
researchers add to our knowledge of education? How much of what they 
contribute is due to their having been in a particular research train- 
ing program, and how much is due to other factors such as their own 
abilities, interests, personality, other training, and career exper- 
iences and opportunities? To answer the first question, we could 
devise a measure of educational research quality (such as the rating 
scales I developed at the BASR (1971) , and use it to evaluate the 
research produced by former trainees. To answer the second question, 
we could develop a causal model of factors related to outstanding 
research, and use path 'analysis or multiple regression analysis to 
ascertain the relative importance of the various factors in explaining 
research contributions. (I am currently working on some models of 
this type, although they are not specifically for trainees) . 



